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Abstract 

Numerous  medical  team  training  programs  have  been  developed  and  implemented 
in  response  to  the  patient  safety  crisis  highlighted  by  the  Institute  of  Medicine. 

The  role  of  effective  teamwork  in  accomplishing  complex  tasks  is  well  accepted 
in  many  domains.  Similarly,  there  is  some  evidence  that  outcomes  in  health  care 
may  depend  on  effective  team  performance.  This  paper  reviews  the  evidence  base 
for  two  categories  of  medical  team  training:  simulator-based  programs  and 
classroom-based  programs.  Specifically,  we  examine  the  purpose  and  strategy  of 
each  and  then  review  the  reported  empirical  evidence.  In  addition,  for  three  of 
four  classroom-based  programs  we  report  the  results  from  a  series  of  course 
observations,  curriculum  reviews,  instructor  interviews,  and  an  independent 
assessment  of  participant  reactions.  Finally,  on  the  basis  of  the  evidence  reviewed, 
we  present  a  set  of  recommendations  for  how  the  health  care  community  can 
develop  medical  team  training  in  the  future. 


Introduction 

Throughout  the  health  care  community,  small  groups  of  individuals  work 
together  as  teams.  Physicians,  nurses,  phannacists,  technicians,  and  other  health 
professionals  must  coordinate  their  activities  to  make  safe  and  efficient  patient 
care  a  priority.  However,  even  though  a  myriad  of  the  conditions  addressed  by 
health  professionals  require  interdisciplinary  teams,  members  of  these  teams  are 
rarely  trained  together,  and  they  often  come  from  separate  disciplines  and  diverse 
educational  programs. 

Given  the  interdisciplinary  nature  of  the  work  and  the  necessity  of  cooperation 
among  the  workers  who  perfonn  it,  it  is  likely  that  teamwork  plays  an  important 
role  in  ensuring  patient  safety  and  avoiding  errors.  Teams  make  fewer  mistakes 
than  do  individuals,  especially  when  each  team  member  knows  his  or  her 
responsibilities,  as  well  as  those  of  other  team  members.  However,  simply 
installing  a  team  structure  does  not  automatically  ensure  it  will  operate 
effectively.  Teamwork  is  not  an  automatic  consequence  of  placing  people 
together  in  the  same  room;  it  depends  on  a  willingness  to  cooperate  toward  shared 
goals.  In  health  care,  shared  goals  might  include  maintaining  a  patient’s  health 
status  and  avoiding  errors. 

In  the  Agency  for  Healthcare  Research  and  Quality  (AHRQ)  Evidence  Report, 
Making  Health  Care  Safer:  A  Critical  Analysis  of  Patient  Safety  Practices,  Pizzi 
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et  al.,  focused  specifically  on  Crew  Resource  Management  (CRM),  a  sub-domain 
of  team  training.4  These  researchers  concluded  that  the  application  of  CRM  in 
medicine  has  tremendous  potential  based  on  its  success  in  aviation.  However, 
they  cautioned  that  additional  research  is  required  to  establish  an  evidence  base 
for  this  health  care  strategy. 

During  the  last  few  years,  the  medical  field  has  developed  several  medical- 
team  training  (MTT)  programs,  some  implemented  in  the  military  and  some 
developed  for  civilian  medicine.  Some  of  these  programs  are  specialty-specific 
(e.g.,  anesthesia),  whereas  others  are  multidisciplinary.  Some  rely  heavily  on 
state-of-the-art  simulators,  whereas  others  primarily  use  classroom  techniques. 
Despite  these  differences,  all  are  heavily  inspired  by  CRM  and  share  the  common 
goal  of  reducing  the  number  of  medical  errors  via  the  application  of  teamwork- 
skills  training. 

The  following  discussion  compares  the  purpose,  strategy,  and  effectiveness  of 
two  distinct  categories  of  MTT :  those  that  are  primarily  simulator-based  and  those 
that  are  primarily  classroom-based.  This  discussion  expands  the  evidence 
presented  by  Pizzi  et  al.4  by  providing  a  detailed  description  of  currently  available 
programs.  Moreover,  we  overview  the  recent  results  from  an  independent  case 
study,  conducted  by  Baker  et  al.,  involving  three  classroom-based  programs.5 
Data  collected  from  MTT  course  observations,  participant  questionnaires,  and 
instructor  interviews  are  reported.  Finally,  we  summarize  the  state-of-the-science 
of  team  training  and  propose  a  series  of  research-based  propositions  for 
improving  the  future  of  MTT. 


Simulator-based  programs 

We  identified  two  MTT  simulator-based  programs:  Anesthesia  Crisis 
Resource  Management  (ACRM)6  and  Team-oriented  Medical  Simulation 
(TOMS).  '  Both  of  these  programs  rely  heavily  on  patient  simulators  to  train 
specific  teamwork  skills  to  physicians  and  other  health  professionals.  They 
emphasize  what  the  Federal  Aviation  Administration  (FAA)  defines  as  the  “Skills 
Practice  and  Feedback”  phase  (i.e.,  Phase  II)  of  a  CRM  training  program. 

Anesthesia  Crisis  Resource  Management  (ACRM) 

The  value  of  ACRM  resides  in  its  realistic  enactment  using  scenarios  of 
operating  room  (OR)  incidents  followed  by  rapid  cycle,  learner-centered 
debriefings  using  videotapes  of  the  clinical  team’s  performance.  Developed  by 
David  Gaba  and  his  colleagues  at  Stanford  University  and  the  Palo  Alto  Veteran 
Affairs  (VA)  Medical  Center,  ACRM  is  designed  to  help  anesthesiologists 
effectively  manage  crises  by  working  in  multidisciplinary  teams  that  include 
physicians,  nurses,  technicians,  and  other  medical  professionals.9' 10  To  facilitate 
this  goal,  ACRM  training  provides  trainees  with  critical  incident  case  studies  to 
review.11  In  addition,  ACRM  provides  training  in  technical  skills  as  well  as  team 
knowledge  and  skills  (refer  to  Table  1).  Training  in  the  selected  teamwork  skills  is 
intended  to  enable  trainees  to  learn  from  adverse  clinical  occurrences  and  to  work 
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more  effectively  with  different  leadership,  followership,  and  communication 
styles.6 

ACRM  training  takes  place  in  a  simulated  operating  room  after  the 
completion  of  the  reading  assignments  that  precede  each  module.  The  simulated 
OR  includes  actual  monitoring  equipment,  a  full-patient  simulator,  a  video  station 
for  recording  the  team’s  performance,  and  a  debriefing  room  that  is  equipped  with 
a  variety  of  audiovisual  equipment.  The  full-patient  simulator  incorporates  a 
series  of  complex  mathematical  models  and  pneumatic  devices  to  simulate  a 
patient’s  breathing,  pulse,  heart  and  lung  sounds,  exhaled  CO2,  thumb  twitches  (to 
assess  muscle  recovery  after  administration  of  paralyzing  drugs),  and  other 
physiological  reactions.6, 12 

The  ACRM  curriculum  is  comprised  of  three  full  days  of  simulation  training 
that  occurs  over  the  course  of  a  resident’s  anesthesiology  training.  Day  1  provides 
an  introduction  to  ACRM  principles  and  skills.  Day  2  provides  a  refresher  on 
these  skills.  It  also  analyzes  clinical  events  from  the  perspective  of  the  clinician’s 
technical  and  teamwork  skills  and  from  the  perspective  of  the  organization  as  a 
larger  system.  Day  3  emphasizes  leadership  training,  debriefing  skills,  and 
adherence  to  the  procedures  established  to  deal  with  adverse  clinical  events.  Each 
training  module  is  comprised  of  a  similar  structure:  pre-assigned  readings,  course 
introduction  and  review  of  materials,  familiarization  with  the  simulator,  case 
study  analysis  and  videotape  reviews,  and  six  hours  of  participating  in  simulator 
scenarios.  These  steps  are  followed  by  an  instructor-led  debriefing  and  a  post¬ 
course  data  collection.  Each  scripted  training  scenario  is  approximately  45 
minutes  long  and  each  debriefing  session  lasts  for  about  40  minutes.6 

Several  instructors  are  required  to  run  the  ACRM  training  scenarios.  They 
might  include  a  retired  OR  nurse  who  role-plays  the  circulating  nurse  and  an 
anesthesiologist  instructor  who  role-plays  the  operating  surgeon.  In  addition,  a 
director  monitors  and  records  the  simulation  from  another  room,  communicating 
with  the  instructors  via  two-way  radios.  Throughout  the  simulation,  trainees  rotate 
through  various  roles,  such  as  first  responder,  scrub  technician,  and  observer.6 

ACRM  training,  complete  with  yearly  refresher  training,  is  currently  used  at 
several  major  teaching  institutions  in  the  United  States  and  around  the  world  (e.g., 
Australia,  Israel,  Denmark).  At  some  centers,  ACRM  training  is  offered  for 
experienced  practitioners  as  well  as  for  trainees.  Moreover,  some  malpractice 
insurers  (e.g.,  Harvard  Risk  Management  Foundation)  have  lowered  their  rate 
structure  for  ACRM-trained  anesthesiologists.6 

Team-oriented  Medical  Simulation  (TOMS) 

The  fundamental  difference  between  ACRM  and  TOMS  is  the  number  of 
participants  included  in  training.  Whereas  ACRM  focuses  solely  on  the 
responsibilities  of  the  anesthesia  crew,  TOMS  provides  interdisciplinary  team 
training  to  surgeons,  nurses,  anesthesiologists,  and  orderlies.  The  TOMS 
program,  which  draws  heavily  on  CRM  training  from  commercial  aviation,1  14 
was  developed  at  the  University  of  Basel  in  Switzerland. 
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The  first  hour  of  TOMS  training  consists  of  a  pre-briefing  that  highlights 
relevant  teamwork  concepts  such  as  situational  awareness,  communication,  and 
decisionmaking.  The  second  hour  is  devoted  to  simulated  laparoscopic  and 
anesthetic  procedures,  using  a  life-like  mannequin  with  live  abattoir  organs  to 
simulate  surgery.  The  third  hour  consists  of  a  team-led  debriefing  that  uses 
videotaped  examples  of  the  team’s  own  perfonnance  to  diagnose  problems  and 
identify  strategies  for  improvement.14 

Simulator-based  program  effectiveness 

A  limited  number  of  studies  were  undertaken  to  investigate  the  effectiveness 
of  simulator-based  training.  Of  the  studies  that  have  been  reported,  trainee 
reactions  to  the  simulation  and  team  behaviors  during  the  simulation  have  served 
as  measures  of  effectiveness. 

Regarding  training  reactions,  of  the  thousands  who  have  undergone  ACRM 
training,  the  majority  have  evaluated  it  favorably.  Even  the  “death  scenario,” 
which  is  specifically  designed  to  assess  how  trainees  handle  a  dying  patient, 
yields  positive  reactions.  Participants  have  been  shown  to  believe  that  ACRM 
contributes  significantly  to  the  safe  practice  of  anesthesiology,  and  these  beliefs 
have  been  shown  to  last  for  up  to  six  months  after  training.  Positive  responses 
have  been  found  to  last  for  up  to  six  months  after  training.6  Similar  participant 
reactions  have  been  reported  for  TOMS,  though  these  results  are  based  on  far 
fewer  TOMS-trained  individuals. 

Regarding  team  behavior,  ACRM  and  TOMS  purport  to  evaluate  a  variety  of 
team  skills  using  trained  raters  who  assess  specific  behaviors  representing  each 
skill  that  was  taught  in  the  curriculum.13' 15  Using  a  rating  scale,  trained  raters 
compile  their  behavioral  observations  and  assign  a  performance  rating  to  each 
behavioral  skill.13' 15  For  ACRM,  rwg  values  (measures  of  inter-rater  agreement) 
ranged  between  .60  and  .93 l5;  an  rwg  of  .70  is  considered  sufficiently  high  to 
reflect  a  satisfactory  degree  of  agreement  among  raters.16 

Despite  these  positive  assessments,  to  our  knowledge  no  studies  have  taken 
the  next  logical  step  of  directly  investigating  the  link  between  team  process  and 
patient-safety  criteria.  In  fact,  virtually  no  research  has  tested  the  effect  of  any 
aspect  of  simulator-based  training  on  actual  performance  outcomes.  This  lack  of 
outcome -related  validity  derives,  at  least  in  part,  from  the  difficulties  associated 
with  quantifying  the  perfonnance  of  physicians  and  other  health  workers.15 

However,  with  respect  to  assessing  the  effects  of  team  process,  the  lack  of 
outcome -related  validity  cannot  be  explained  so  easily  because  programmed 
outcomes  are  embedded  into  the  simulator  training  scenarios  (e.g.,  the  “death 
scenario”).  Thus,  we  believe  that  developing  measures  to  assess  the  effectiveness 
of  teamwork  in  facilitating  positive  outcomes  and  in  successfully  managing,  if  not 
avoiding,  negative  outcomes  would  constitute  a  constructive  focus  for  future 
research.  Furthermore,  given  the  current  state  of  simulation,  devising  training 
scenarios  for  which  the  outcome  is  contingent  upon  the  level  of  trainees’ 
demonstrated  teamwork  skills  might  also  be  worthwhile.  The  FAA-sponsored 
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Advanced  Qualification  Program  (AQP)  has  adopted  this  strategy.  Airline 
captains  and  first  officers  are  certified  to  fly  during  a  line  operational  evaluation 
(LOE)  upon  attainment  of  a  full  complement  of  technical  and  CRM  skills. 


Classroom-based  programs 

We  identified  four  MTT  classroom-based  programs — MedTeams™,  Medical 
Team  Management  (MTM),  Dynamic  Outcomes  Management'”’  (DOM,  renamed 
Lifewings™),  and  Geriatric  Interdisciplinary  Team  Training  (GITT) — from  a 
comprehensive  review  of  the  literature  (refer  to  Baker  et  al.  for  a  detailed 
explanation  as  to  how  this  search  was  conducted).17  These  programs  rely 
primarily  on  classroom-based  instructional  strategies  (e.g.,  lectures,  video 
demonstrations,  role  plays,  etc.)  and  focus  on  what  the  FAA  defines  as  the 
“Awareness”  phase  (i.e.,  Phase  I)  of  CRM  training.  However,  each  course  makes 
provisions  for  follow-up  skills  practice  (i.e.,  Phase  II)  and  recurrency  training 
(i.e.,  Phase  III).  In  the  cases  of  MedTeams,  MTM,  and  DOM,  we  were  able  to 
conduct  in-class  observations  and  instructor  interviews.  We  also  collected 
information  on  student  reactions.  Below,  we  describe  the  purpose  and  strategy  of 
each  program  and  provide  information  on  the  effectiveness  of  classroom-based 
training  based  on  the  available  research  and  our  findings  from  our  case  study 
investigation.5 

MedTeams™ 

The  primary  purpose  of  MedTeams  is  to  reduce  medical  errors  through 
interdisciplinary  teamwork.  MedTeams  was  initially  developed  by  Dynamics 
Research  Corporation  (DRC)  for  emergency  departments  (EDs)  on  the  premise 
that  most  errors  result  from  breakdowns  in  systems-level  defenses  that  occur  over 
time.  According  to  the  MedTeams  ED  curriculum,  each  team  member  has  a 
vested  interest  in  maintaining  patient  safety  and  is  expected  to  take  an  assertive 
role  in  breaking  the  error  chain.  MedTeams  defines  a  core  ED  team  as  a  group  of 
3-10  (average  =  6)  medical  personnel  who  work  interdependently  during  a  shift 
and  who  have  been  trained  to  use  specific  teamwork  behaviors  to  coordinate  their 
clinical  interactions.  Each  core  team  includes  at  least  one  physician  and  one  nurse. 
A  coordinating  team,  tasked  with  assigning  new  patients  to  the  core  teams  and 
providing  additional  resources  as  necessary,  manages  several  core  teams. 

MedTeams  training  was  developed  from  an  evaluation-driven  course  design. 
Based  on  needs-analysis  data,  DRC  identified  five  critical  dimensions  that  were 
necessary  for  effective  teamwork  (Table  1).  Then,  they  identified  48  specific, 
observable  behaviors  that  were  linked  to  these  dimensions  and  constructed 
Behaviorally  Anchored  Rating  Scales  (BARS)19  for  each  behavior.  Finally,  to 
establish  its  content  validity,  they  reviewed  and  refined  the  curriculum  during 
three  five-day  expert  panel  sessions  that  included  ED  physicians  and  nurses  from 
12  hospitals  of  various  sizes.20  Expert  panel  review  and  modification  of  the 
curriculum  has  been  used  to  create  Labor  and  Delivery  (L&D)  and  Operating 
Room  (OR)  versions  of  MedTeams. 
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MedTeams  uses  a  train-the-trainer  approach  to  implement  the  training. 
Individuals,  designated  by  their  facility,  receive  comprehensive  training  on  how  to 
teach  MedTeams  and  are  certified  as  MedTeams  instructors.  The  course  consists 
of  an  eight-hour  block  of  classroom  instruction  that  contains  an  introduction 
module,  five  learning  modules,  and  an  integration  unit.  After  completing  the 
classroom  training,  DRC  reports  that  each  team  member  participates  in  a  four- 
hour  practicum  that  involves  practicing  teamwork  behaviors  and  receiving 

feedback  from  a  trained  instructor.  Coaching,  mentoring,  and  review  sessions  are 

20 

also  provided  during  regular  work  shifts. 

Table  1.  Team  knowledge  and  skills  taught  in  MedTeams™,  DOM,  and  MTM 


MedTeams™ 

DOM 

MTM 

Maintaining  team  structure 
and  climate 

Team  management 

Situation  awareness 

Problem-solving  skills 

Recognizing  adverse 
events 

Operating  strategy 

Execution  of  plans  and 
management  of  work  load 

Communications 

Communication 

Communication  skills 

Decision  making 

Command  authority 

Team  improvement  skills 

Distribution  of  workload 

Workload  performance 

Knowledge  of  the 
components  of  teamwork 

Debriefing 

Resources 

Situation  awareness 

Policy/regulation 

DOM=Dynamic  Outcomes  Management 
MTM=Medical  Team  Management 


Medical  Team  Management  (MTM) 

The  Air  Force  developed  MTM  training  specifically  in  response  to  an  incident 
at  an  Air  Force  facility  in  which  poor  teamwork  led  to  a  neurologically  impaired 
newborn.  Similar  to  MedTeams,  the  primary  purpose  of  MTM  is  to  reduce 
medical  errors,  in  this  case  by  teaching  human-factors  concepts  to 
interdisciplinary  teams  of  medical  professionals.  ’  A  secondary  purpose  is  to 
change  the  military’s  traditional  medical  culture,  which  focuses  on  individual 
performance,  an  emphasis  that  creates  communication  barriers.  In  contrast,  MTM 
specifically  fosters  a  culture  that  values  team  performance  and  encourages 
effective  communication.  ~  Its  theoretical  position  is  that  this  new  culture  will 
facilitate  teamwork,  thereby  reducing  errors. 

The  MTM  training  program  has  two  major  components:  a  three-day  train-the- 
trainer  course  and  a  medical  treatment  facility  course.  Upon  completing  the  train- 
the-trainer  course,  graduates  return  to  their  respective  medical  facilities  to  train 
the  remaining  staff  in  teamwork  principles. ~  The  MTM  curriculum  includes  an 
introduction  to  the  program,  overviews  of  key  patient  safety  and  CRM  issues,  and 
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specific  modules  for  seven  team  skills  (Table  1).  Case  studies,  vignettes,  and  tools 
are  interspersed  throughout  the  curriculum  to  reinforce  the  importance  of  effective 
teamwork. 

In  2001,  the  Air  Force  Surgeon  General  mandated  MTM  training  for  all  high- 
risk  specialties:  emergency  departments,  operating  rooms,  obstetric  departments, 
intensive  care  units,  and  neonatal  care  units.'  As  of  February  2003,  over  2,000 
medical  treatment  facility  personnel  have  received  MTM  training.  Formal 
evaluation  has  yet  to  be  done  on  the  effectiveness  of  MTM. 

Dynamic  Outcomes  Management  (DOM) 

(currently  Lifewings™) 

The  primary  purpose  of  DOM  is  to  increase  patient  safety,  reduce  medical 
errors,  and  improve  the  quality  of  health  care  by  improving  trainees’  skills  in 
team-building,  recognizing  adverse  situations,  counteracting  the  effects  of  stress 
and  fatigue,  communicating,  and  decisionmaking.  DOM  provides 
interdisciplinary  team  training  to  surgeons,  nurses,  and  anesthesiologists.  The 
program  draws  heavily  on  CRM  training  from  aviation"  and  was  developed  by 
Crew  Training  International  (CTI).  CTI  (http://www.cti-cnn.com)  recently 
renamed  DOM  Lifewings™  (http://www.saferpatients.com). 

DOM,  which  is  quite  similar  to  MedTeams  and  to  MTM,  includes  8  hours  of 
classroom-based,  interactive  training  that  incorporates  facilitated  discussion,  role 
playing,  case  studies,  behavior  modeling,  and  knowledge  testing.  During  the  8- 
hour  session,  two  highly  trained  CTI  instructors  (typically  former  pilots)  lead 
participants  through  strategies  for  building  an  effective  team.  These  techniques 
include  recognition  of  adverse  situations,  recommendations  for  managing  conflict 
constructively,  guidance  for  mitigating  the  effects  of  stress,  training  in 
decisionmaking  skills,  recommendations  for  providing  effective  perfonnance 
feedback,  and  principles  for  mitigating  the  effects  of  fatigue.  To  reinforce  the 
principles  of  DOM  training,  CTI  developed  a  “challenge  and  response  checklist,” 
which  trainees  are  required  to  use  in  the  OR. 

Geriatric  Interdisciplinary  Team  Training  (GITT) 

The  primary  purpose  of  GITT  is  to  create  a  cadre  of  well-trained  professionals 
who  can  leverage  the  effects  of  interdisciplinary  teamwork  to  improve  geriatric 
patient  care.  To  this  end,  GITT  provides  interdisciplinary  team  training  for 

physicians,  nurses,  nurse  practitioners,  social  workers,  pharmacists,  therapists, 

26 

and  administrators. 

GITT,  which  is  also  quite  similar  to  MedTeams,  MTM,  and  DOM,  includes  a 
full  day  of  team  self-evaluation  and  skills  training.  The  team  self-evaluation 
exercise  uses  the  Strength  Development  Inventory  8  "7  to  help  team  members 
recognize  their  preferred  interpersonal  styles.  It  also  uses  the  Team  Signatures 
Technology®  28  to  help  each  team  identify  their  unique  dynamics  through 
describing  the  team’s  level  of  cohesion,  leadership,  diversity,  and  other  relevant 
characteristics.  Following  the  self-evaluation  exercises,  the  team  members  receive 
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classroom  instruction  in  the  principles  of  effective  teamwork,  phases  of  team 
development,  conflict  management,  leadership,  and  other  factors.26  A  half-day  of 
refresher  training  is  provided  approximately  one  year  later. 

Data  concerning  the  development  and  implementation  of  GITT  are  limited. 

Of  the  original  eight  teams  that  participated  in  GITT  (all  of  which  were  from 
geriatric  treatment  facilities  in  Rhode  Island),  only  three  participated  in  the 
follow-up.  The  remaining  five  teams  had  ceased  to  exist  in  their  original 
configurations  because  of  administrative  reassignments. 

Classroom-based  program  effectiveness 

Similar  to  simulator-based  programs,  few  studies  have  investigated  the 
effectiveness  of  classroom-based  MTT.  An  extensive  review  of  the  literature 
produced  one  study  on  MedTeams — while  another  is  ongoing — and  one  study  on 
DOM  effectiveness.17  Because  data  on  the  effectiveness  of  these  programs  are 
limited  and  the  studies  to  date  have  been  conducted  by  the  program  developers, 
AHRQ  and  the  DoD  tasked  the  American  Institutes  of  Research  (AIR)  with 
conducting  a  case  study  analysis  of  the  three  most  widely  applied  classroom- 
based  programs:  MedTeams,  MTM,  and  DOM.  Here  we  review  the  empirical 
evidence  for  MedTeams  and  DOM,  and  then  provide  a  summary  of  our  case  study 
findings. 


Empirical  evidence 

The  one  reported  evaluation  of  the  MedTeams  approach  was  conducted  in  the 
ED,  while  the  one  underway  is  being  conducted  in  L&D.~  The  ED  study 
involved  a  multi-site,  single-crossover,  quasi-experimental  design.30  In  this  study, 
nine  EDs  (6  in  the  experimental  group  and  3  in  the  control  group)  were  observed 
during  a  14-month  interval  that  encompassed  pre-training  baseline  measures,  the 
training  intervention  proper,  and  post-training  evaluations.  A  suite  of  17  process 
and  performance  measures  was  collected.  To  ensure  rating  accuracy,  all 
observation-based  measures  were  collected  by  trained  raters,  and  measures  of 
inter-rater  agreement  were  periodically  calculated  to  ensure  that  the  raters 
remained  calibrated.30  Finally,  because  data  were  clustered,  Generalized 
Estimating  Equations  (GEE)  were  used  to  test  the  effect  of  the  hospital-level 
intervention  using  case-level  data.  The  results  suggested  that  in  contrast  to  the 
control  group,  the  trained  groups  showed  the  following:  that  significant  gains 
were  achieved  in  teamwork-related  knowledge,  skills,  and  attitudes;  that  the 
intervention  did  not  increase  self-reported  task  workload;  and  that  the  error  rate 
decreased  sharply.  However,  these  results  may  be  questioned  because  the  raters 
were  not  blind  to  conditions  and  EDs  were  not  randomly  assigned  to  experimental 
and  control  groups. 

o  1 

The  second  study  is  ongoing  in  L&D  units  in  civilian  and  military  hospitals. 
Unlike  the  previous  study,  in  which  the  EDs  chose  to  participate  in  either  the 
experimental  or  control  conditions,  this  study  was  designed  as  a  Randomized 
Clinical  Trial  (RCT).  Based  on  an  a  priori  power  analysis,  24  hospital  L&D  units 


260 


Medical  Team  Training  Programs 


were  randomly  assigned  to  participate  in  either  the  experimental  or  control 
conditions  (up  to  12  per  condition).  Many  other  aspects  of  the  L&D  study  mirror 
that  of  Simon,  et  al.,  with  multiple  performance  measures  that  focus  on  patient 
outcomes,  team  process,  and  staff  and  patient  satisfaction.29 

Regarding  DOM,  over  160  surgical  staff  members  at  Methodist  University 
Hospital  (Memphis,  TN)  have  completed  DOM  training.  An  evaluation  of  the 
hospital  found  improvements  in  participants’  attitudes  toward  the  importance  of 
teamwork  issues  in  the  OR,  favorable  reactions  concerning  the  usefulness  of 
DOM  training,  and  a  50-percent  reduction  in  the  number  of  surgical  count  errors. 
However,  the  small  sample  size  makes  it  difficult  to  generalize  the  results. 
Moreover,  the  lack  of  a  control  group  makes  it  difficult  to  determine  whether  the 
training  caused  these  improved  outcomes. 


Case  study  results 

In  the  summer  of  2003,  AIR  conducted  independent,  detailed  case  study 
analyses  of  MedTeams,  MTM,  and  DOM.  Case  studies  were  based  upon  the 
following:  reviews  of  student  and  instructor  guides,  slides,  and  other  audio-visual 
materials  that  course  developers  provided;  review  of  published  documents  on  a 
specific  program’s  effectiveness;  observations  of  the  classroom  portion  of  each  of 
the  courses;  and  the  collection  of  pre-training  data  on  student  experiences  and 
expectations  as  well  as  new,  independent  post-training  data  on  student  reactions  to 
MedTeams,  MTM,  and  DOM.  Finally,  for  MedTeams  and  MTM,  trained  AIR 
staff  conducted  one-on-one  interviews  with  instructors  who  had  taught  or  were 
going  to  teach  the  course. 

Collectively,  the  results  suggested  MedTeams,  MTM,  and  DOM  possess 
several  desirable  characteristics.  First,  these  classroom-based  training  programs 
employ  appropriate  adult  and  active  learning  techniques  to  develop  participant 
awareness  of  team-related  skills.  Each  program  uses  a  variety  of  instructional 
media  to  provide  infonnation  about  specific  team  skills  and  to  demonstrate  the 
importance  of  teamwork.  Second,  each  program  takes  an  interdisciplinary,  as 
opposed  to  a  uni-disciplinary,  approach  to  health  care  training.  Typical  classes 
include  a  mixture  of  physicians,  nurses,  technicians,  and  other  heath  care 
professionals.  Third,  our  independent  collection  of  post-training  reactions 
suggests  that  participants  had  positive  reactions  to  MedTeams,  MTM,  and  DOM 
training  (Table  2).  Participants  in  each  class  were  asked  to  independently  rate  the 
extent  to  which  they  agreed  with  nine  statements  about  training  using  a  five-point 
scale  featuring  the  following  ratings:  l=Strongly  Disagree;  3=Neutral;  5=Strongly 
Agree.  In  each  case,  participants  indicated  that  the  training  was  well  organized, 
and  felt  that  they  could  use  many  of  the  strategies  discussed  during  training  upon 
returning  to  their  jobs.  Finally,  in  the  case  of  MedTeams  and  DOM,  program 
developers  are  making  an  effort  to  collect  data  to  demonstrate  that  MTT  has  an 
effect  on  participant  attitudes,  knowledge,  and  skills  and  that  these  intermediate 

29  30 

outcomes  can  be  linked  to  quality  indicators  in  health  care.  ’ 
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Table  2.  Trainee  reactions  to  MedTeams™,  DOM,  and  MTM 


MedTeams™ 

(n=218) 

MTM  (n=26) 

DOM  (n=78) 

Item 

Mean 

Rating* 

Std. 

Dev. 

Mean 

Rating* 

Std. 

Dev. 

Mean 

Rating* 

Std. 

Dev. 

The  training  was 
well-organized. 

4.4 

0.68 

m 

0.49 

4.7 

0.72 

1  am  confident  that 

1  can  perform  the 
tasks  that  were 
trained. 

4.4 

0.58 

0.72 

4.4 

0.70 

1  am  confident  that 

1  understood  the 
training  content. 

m 

0.60 

0.59 

■ 

0.65 

1  am  confident  that 

1  can  use  the 
knowledge  that  1 
learned  on  the  job. 

4.4 

0.69 

0.80 

0.68 

The  training 
content  was 
appropriate  for  my 
department. 

■ 

0.72 

0.75 

4.4 

0.83 

Training  will  help 
my  department 
improve  patient 
safety. 

4.1 

0.81 

4.1 

0.60 

4.4 

0.83 

As  a  result  of  this 
training,  1  feel  more 
confident  about  my 
ability  to  work 
effectively  in  a 
team. 

4.1 

0.83 

■ 

0.65 

■ 

0.73 

Training  prepared 
me  to  work 
effectively  in  my 
job. 

4.0 

0.82 

4.0 

0.71 

4.4 

0.78 

Training  was  an 
effective  use  of  my 
time. 

3.9 

0.95 

4.1 

0.70 

4.4 

0.86 

‘Rating  scale:  1=Strongly  Disagree;  2=Disagree;  3=Neutral;  4=Agree;  5=Strongly  Agree 

DOM=Dynamic  Outcomes  Management 
MTM=Medical  Team  Management 
Std.  Dev.=Standard  Deviation 

Note:  Response  rate  for  all  courses  exceeded  90%. 

Nevertheless,  classroom-based  programs  were  found  to  have  a  number  of 
limitations.  For  example,  except  for  the  ED  version  of  MedTeams,  none  of  the 
programs  was  based  on  a  comprehensive  pre-training  needs  analysis.  Needs 
analysis  is  a  critical  first  step  in  development  of  training  methods.  Such  an 
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analysis  uncovers  the  specific  training  requirements  as  well  as  cultural  and  other 
organizational  issues  that  can  impact  training.32  Second,  even  though  MedTeams 
and  MTM  provide  instructors  with  comprehensive  instructor  training,  our 
observations  found  that  trained  instructors  demonstrated  great  variability  in  the 
way  they  conducted  training.  For  example,  regarding  observations  of  MedTeams, 
classes  ranged  from  three  to  seven  hours,  and  the  quality  of  this  instruction  varied 
greatly.  Third,  and  quite  contrary  to  the  positive  reactions,  several  of  the 
instructors  and  students  we  interviewed  said  that  there  was  only  a  40-50  percent 
chance  of  MTT  being  implemented  successfully.  Their  major  concerns  were  that 
the  individuals  in  the  operating  room  do  not  have  enough  time  to  conduct  ongoing 
training,  and  that  the  length  of  the  training  course  and  the  need  for  refresher 
training  make  it  prohibitive.  Finally,  all  of  the  classroom-based  programs 
presented  limited  opportunities  for  participants  to  receive  structured  practice  and 
feedback  on  critical  teamwork  skills.  Moreover,  role-plays  were  the  primary 
instructional  strategy  used  for  skills  practice.  Although  role-playing  has  been 
shown  to  enhance  specific  team  skills1,  this  strategy  seems  limited  because  of  the 
practical  constraints  associated  with  the  classroom  environment  and  the  lack  of 
patient  or  patient  proxy  interaction  (i.e.,  the  ability  to  allow  all  class  members  the 
opportunity  to  receive  practice  and  feedback  during  a  role-play). 


Conclusions 

In  summary,  few  would  dispute  the  importance  of  teamwork  in  promoting 
safe  and  efficient  health  care.  Team  training  began  with  the  introduction  of 
ACRM  in  anesthesiology  and  recently  proliferated  with  the  publication  of  To  Err 
Is  Human:  Building  a  Safer  Health  System ,33  In  this  paper,  we  reviewed  six  MTT 
programs  that  have  been  implemented  in  a  variety  of  health  care  settings.  We 
believe  development,  implementation,  and  evaluation  of  such  programs  are 
critical  to  the  evolution  of  both  the  science  and  the  practice  of  MTT.  The  efficacy 
of  CRM  has  been  established,  in  part,  because  CRM  has  been  evaluated 
throughout  its  evolution.  CRM  training  was  developed  interactively — introducing 
and  testing  the  effectiveness  of  different  strategies — which  allowed  for  the  best 
possible  results.34 

We  believe  that  the  medical  community  could  significantly  improve  MTT  by 
looking  to  CRM  as  well  as  other  domains  where  team  training  strategies  have 
been  developed  and  have  enjoyed  great  success.  In  particular,  the  Tactical 
Decision-Making  Under  Stress  (TADMUS)  project,  conducted  in  the  surface 
communities  of  the  U.S.  Navy,  produced  a  number  of  useful  tools  and  lessons 
learned  that  are  applicable  to  health  care.  Based  on  our  review  of  MTT 
programs  and  the  extant  literature,  we  advocate  the  following  recommendations. 

First,  we  recommend  that  the  health  care  community  develop  a  standard  set  of 
generic  teamwork-related  knowledge,  skill,  and  attitude  competencies.  Parry 
defined  the  term  “competency”  as  a  cluster  of  related  knowledge,  skills,  and 
attitudes  that  (1)  affects  a  major  part  of  one’s  job  (i.e.,  one  or  more  key  roles  or 
responsibilities);  (2)  correlates  with  successful  job  performance;  (3)  can  be 
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measured  against  well-accepted  standards;  and  (4)  can  be  improved  through 
training  and  development.  Team  knowledge,  skill,  and  attitude  competencies 
would  represent  the  core  elements  of  successful  teamwork  in  health  care.  A 
review  of  the  MedTeams,  MTM,  and  DOM  showed  that  many  of  the  principles 
that  are  advocated  and  behaviors  that  are  taught  are  similar  across  programs. 
However,  each  program  advocates  somewhat  different  team  knowledge  and  skills, 
and  these  attributes  are  often  at  different  levels  of  specification.  We  believe  that 
developing  a  core,  agreed-upon  list  that  confonns  to  Parry’s  definition  of 
competency  would  be  a  significant  step  forward  for  health  care.  It  would  reduce 
potential  confusion  as  well  as  begin  to  establish  a  common  language  for 
describing  teamwork  in  health  care. 

Second,  we  recommend  that  instructional  designers  look  beyond  aviation 
CRM  training  and  leverage  all  available  research  and  tools  (e.g.,  the  tremendous 
amount  of  U.S.  Navy  research  on  teams)  when  developing  MTT  programs.  For 
example,  Salas  and  his  colleagues  have  compiled  an  extensive  collection  of 
principles  and  guidelines  for  assertiveness  training  ,  cross-training  ,  stress 
management  training  ,  and  team  self-correction. 

Unfortunately,  the  existing  medical  team  training  programs  do  not  appear  to 
have  leveraged  this  body  of  research.  For  example,  the  MTT  programs  that  we 
reviewed  rely  almost  exclusively  on  classroom-based  or  simulator-based  training 
methods,  rather  than  on  choosing  from  a  variety  of  instructional  strategies  to 
complement  the  specific  training  content.  With  few  exceptions,  new  advances  in 
training  technology — such  as  computer-based  training,  low-fidelity  simulations, 
standardized  patients,  embedded  training,  and  scenario-based  training — have 
rarely  been  used,  despite  growing  evidence  regarding  their  effectiveness.40  Recent 
advances  in  the  training  theory — such  as  the  effect  of  pre-  and  post-training 
factors  on  training  outcomes,  the  effect  of  practice  schedules  on  skills  acquisition 
and  retention,  and  the  critical  role  of  individual  differences  in  shaping  trainees’ 
motivation — have  similarly  been  ignored.40  43 

Third,  in  addition  to  gaining  traction  from  the  available  research  on  team 
training,  we  recommend  that  future  MTT  programs  address  all  three  phases  of  a 
comprehensive  team  training  program:  awareness,  skills  practice  and  feedback, 
and  recurrence.  This  approach  has  been  a  major  factor  in  the  success  of  CRM  as 
well  as  in  the  integration  of  team  skills  training  throughout  pilot  professional 
training.  Certainly,  one  place  for  health  care  to  start  would  be  to  combine  best 
practices  from  classroom-based  (awareness  phase)  and  simulator-based  training 
(skills  practice  and  feedback).  We  recognize  that  programs  like  MedTeams, 

MTM,  and  DOM  make  provisions  for  skills  practice,  but  the  addition  of 
simulator-based  training  (either  using  low  or  high  fidelity  simulations)  would 
likely  be  beneficial. 

Finally,  we  recommend  that  AHRQ  develop  advisory  circulars — much  like 
the  Clinical  Practice  Guidelines,  which  are  developed  to  treat  specific  medical 
conditions — on  issues  related  to  team  training  and  error  prevention.  We  believe 
that  human  factors-related  advisory  circulars  would  go  a  long  way  toward 
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educating  the  medical  community  about  the  importance  of  MTT  for  ensuring 
patient  safety  and  consistency  across  MTT  programs. 
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